Using a script-based program keeps a record of everything you do

R is a script-based language. You write down a list of instructions and it will follow, performing one action after another. This is different to ‘point and click’ software like Microsoft Excel, and it can feel a bit cumbersome.

In Excel, you can perform a series of steps:

• Open a file.
• Delete column that you don’t need.
• Select all of your data and sort from highest to lowest.
• Delete all the rows that have missing values.
• Save your file.

But, there is no record of any of this. You might look back at the data in a week’s time and not know what rows you have deleted. This is particularly important when you go to write up your metholodolgy for an assignment or thesis. You might not be able to recreate your own work.

R is a script-based program. You write a list of instructions and R will follow it. This is wonderfully handy:

• everything you have told it to do, it will do;
• you have recorded everything you have told it to do.

Using a free, open-sourced program is beneficial

There are many proprietary script-based programs for analysing data: Stata, SAS, Eviews, SPSS, Matlab. I imagine you will encouter a few of these during your undergrad.

How to install R and R Studio

You will need to download and install R and R Studio. (And you can skip this subsection if you already have!).

R is the language and the program. Think of it as the engine that powers the things you do. You can download it for:

• Mac: https://cran.rstudio.com/bin/macosx/R-3.5.2.pkg

• Windows: https://cranrstudio.com/bin/windows/base/R-3.5.2-win.exe

Once downloaded, follow the prompts to install. Restart your computer if required.

R Studio is the interface you will use R with. The technical term is an ‘integrated development environment’ (IDE) for R. Think of it as the dashboard that shows you all the things you’ve got going on in R. You can download it for:

• Mac: https://download1.rstudio.org/RStudio-1.1.463.dmg

• Windows: https://download1.rstudio.org/RStudio-1.1.463.exe

Then, follow the prompts to install and restart if required.

R Studio layout

R Studio is an integrated development environment (IDE) and is how we will interact with R. It looks like this:

The four panes are labelled in \(\color{green}{\text{green}}\) :

• Top left: your script. This is where you will write your code.
• Bottom left: your console. This is where your code will be sent to, and where some of your output will show. You can also write code directly into the console; but there will be no record of it in your script!
• Top right: your environment. This lists all the things that R knows about so far. If you define an object, it will appear here.
• Bottom right: a bit of a miscellaneous pane. Files list the files in your project. Packages lists the packages you have installed and loaded. Viewer is where our charts will be displayed.

I know this can all look a bit intimidating the first time you see it. That’s okay! We’ll get to know R Studio more as we go through this course.

Good folder structure and R Projects

Good folder structure is tedious and abstract and not-at-all-fun but it makes everything in the future easier. It simply means you have:

1. One folder for each project. Here, the example is introduction_to_R. Your projects might be something like econometrics_assignment2 or honours_thesis. Whatever the project, everything you need for it is contained within the main project folder.
2. A set of standardised subfolders. For example, keep the data you are using in a folder called data. Keep output (tables, charts, etc) in an output folder. Note that you can set these up how you like: but consistency makes it easier for you to switch between projects and collaborate with others.

Your script will often ask for things on the computer. For example, “read in this dataset” or “save this chart to a place”. For that, we have to tell the computer where it is. We can do that in two ways: 1) by explicitly setting a working directory (which is not the best way), and 2) by setting up a R Project (which is the best way to do it).

setwd("Users/wfmackey/Documents/myRfolder/this_project_of_mine")

Comments in your code

‘Comments’ are notes that live in your code and are preceded with #. R will ignore anything after the # symbol:

# This is a good comment
12 + 2 # Those are great numbers

## [1] 14

If we forgot to use the # to ‘comment’ things, we would generate a bunch of errors as R tries to work out what you’re on about:

This is a good comment
12 + 2 Those are great numbers

## Error: <text>:1:6: unexpected symbol
## 1: This is
##          ^

Objects

Objects are ‘things’ that R knows about. They can be simple as a single number, or more complicated like an entire dataset.

You can tell R about an object using the assign <- operator:

# Define 'fave_number' as 12
fave_number <- 12

We have said: take the number 12 and assign it <- to the object fave_number. R will note this down, put fave_number in our environment and we will be able to use it when we want later. Like:

# Multiply 'fave_number' by 2
fave_number * 2

## [1] 24

Or:

# Add 37 to 'fave_number'
fave_number + 37

## [1] 49

We can have lots of objects stored in our environment, and we can call them whenever we want (after we have defined them):

least_fave_number <- 5
fave_number + least_fave_number

## [1] 17

Functions

A function takes inputs (aka ‘arguments’) and produces outputs.

We can use the c function to combine (concatenate) numbers into a series of numbers (a vector):

c(3, 4, 5)

## [1] 3 4 5

The output above, like all output in this document, is preceded by ## and then [1], meaning the first line of output. Here, the output is the vector of numbers we entered into the c function.

We can also nest functions, meaning we have one function inside another function. For example, we can combine numbers into a vector using the c function, then we can take the average (mean) of the vector:

# Use the c function to combine numbers (input) into a vector (output)
# Then take the mean of that vector:
mean(c(3, 4, 5))

## [1] 4

But nested functions are a bit difficult to read. You have to start from the inside and read outwards. Alternatively, we could assign our vector to an object using the assign <-operator:

# Use the c function to combine numbers (input) into a vector (output)
# And assign that to the object 'goodnumbers'
goodnumbers <- c(3, 4, 5)

# Then take the mean of goodnumbers
mean(goodnumbers)

## [1] 4

This will make changes in our environment to the top-right: it adds the object goodnumbers.

It will also produce output in the console to the bottom-left: the mean of goodnumbers.

It should look something like this:

Packages

R uses functions to do things. A package is a collection of functions. Some packages are installed when you install R, like base and stats (this is called “base R”).

A wonderful benefit of R is being able to use the community’s collection of functions. The tidyverse is a collection of functions that make data wrangling and visualisation much easier. And we can use this package for free.

Installing a package is like installing an app on your phone or computer: you need to do it, and you only need to do it once.

You can install a package using the install.packages function. Note that there will be lots of text that appears when installing a package. This text doesn’t make great reading, but it lays out what the package is doing as it installs. If there is an error installing, this is where you’ll find some (hopefully) useful infromation about why.

install.packages("tidyverse")

Now we need to load the package using the library function, like opening an app you have installed. We do this every time (every ‘session’) we want to use it.

library(tidyverse)

You will make lots of errors and this will be frustrating

R will follow your instructions to the tee. If you misspelll something or put an argument where it doesn^t belong, R will try to do the thing you asked and it will fail. Hopefully, it will clearly tell you what happened; other times it will be vague.

This means you will spend a lot of time debugging your code: you think what you’ve written makes sense, but you get an error. You try to work out what’s gone wrong; you play around; you search the internet; you give up on R; you go outside and enjoy your R-free life;

Make sure your brackets add up

Peeking at the data

Much like Excel, we can explore the raw gapminder dataset with our eyes.

View will open up a new tab that displays your dataset. You can scroll through it and look at each row and variable in your data.

View(gapminder)

If we just want a quick look, head will print just the first few observations. This is handy to check on things as you’re going along.

head(gapminder)

## # A tibble: 6 x 6
##   country     continent  year lifeExp      pop gdpPercap
##   <chr>       <chr>     <dbl>   <dbl>    <dbl>     <dbl>
## 1 Afghanistan Asia       1952    28.8  8425333      779.
## 2 Afghanistan Asia       1957    30.3  9240934      821.
## 3 Afghanistan Asia       1962    32.0 10267083      853.
## 4 Afghanistan Asia       1967    34.0 11537966      836.
## 5 Afghanistan Asia       1972    36.1 13079460      740.
## 6 Afghanistan Asia       1977    38.4 14880372      786.

names will display the names of all variables in the dataset (and is often the answer to ‘what was that variable called again…’)

names(gapminder)

## [1] "country"   "continent" "year"      "lifeExp"   "pop"       "gdpPercap"

Our first ggplot

So we can take the gapminder dataset, generate an empty plot using ggplot and fill it with a point: lifeExp on the y axis, and gdpPercap on the x axis:

  ggplot(data = gapminder) +
  geom_point(mapping = aes(x = lifeExp,
                           y = gdpPercap))

What just happened there? We used the ggplot2 package, which stands for the layered grammar of graphics plot⁵

Our plot has a few components:

• Data: our gapminder object.
• An ggplot using the ggplot function.
• A geometric object: we have chosen geom_point to plot dots.
  • Aesthetic mapping: within geom_point we have defined an aesthetic with aes and mapped the x axis to represent lifeExp, and the y axis to represent gdpPercap.

We can see how that plays out by creating an empty plot:

  ggplot()

Then fill it with out data:

  ggplot(gapminder)

And then we have to explain how to map the data to the plot using aesthetics aes, which we add to the plot using + (noting that the + comes at the end of a line).

  ggplot(data = gapminder) +
  geom_point(mapping = aes(x = lifeExp,
                           y = gdpPercap))

Start with an empty plot, then layer on a geometric object geom with some aesthetics aes(). We can add more layers to the same plot.

Changing scales

Our plots so far have been on ‘linear’ scales: they go up by the same amount for the whole scale.

But this might not always be the right scale for things that happen exponentially. For example, we might expect that income grows much faster than health does. So we decde to examine the relationship between (exponetial) GDP per capita and life expectancy. We want to use a log10 scale on the y axis (where income is); so we add + the scale_y_log10 function to our plot:

  ggplot(data = gapminder,
         mapping = aes(x = lifeExp,
                       y = gdpPercap)) +
  geom_point() +
  scale_y_log10()

Adding a trendline

To add another geom to the same plot, we use the and + symbol, then add our new set of instructions on a new line. A trendline is called using geom_smooth:

  ggplot(data = gapminder) +
  geom_point(mapping = aes(x = lifeExp,
                           y = gdpPercap)) +
  geom_smooth(mapping = aes(x = lifeExp,
                           y = gdpPercap)) +
  scale_y_log10()

Note: there is a whole library of geoms to explore at https://ggplot2.tidyverse.org/reference/.

The trendline helps us with our question: we can see that, overall, the higher income is correlated with higher life expectancy. This is a result we would expect.

But the way we are looking at this data might be hiding some important insights. We should explore them.

Adding colour

Our scatter plot so far shows two aesthetics (aes): lifeExp mapped to the x axis, and gdpPercap mapped to the y axis.

Let’s look at our variable names again and explore if we can squeeze some more information out of this plot:

names(gapminder)

## [1] "country"   "continent" "year"      "lifeExp"   "pop"       "gdpPercap"

It might be interesting to see how things vary by continent. So let’s map continent to the colour aesthetic. We do this in the same way we mapped life expectancy and GDP per capita to the x and y axes:

  ggplot(data = gapminder) +
  geom_point(mapping = aes(x = lifeExp,
                           y = gdpPercap,
                           colour = continent)) +
  geom_smooth(mapping = aes(x = lifeExp,
                            y = gdpPercap,
                            colour = continent)) +
  scale_y_log10()

We have repeated ourselves a bit here. For each geom we have set the aesthetics: “this is the x axis”, “this is the y axis” and “this is the colour”. To save ourselves some typing (and ensure we’re being consistent)lh we can set the aesthetics in the first ggplot function. The geoms that follow will ‘inherit’ these aesthetics:

  ggplot(data = gapminder,
         mapping = aes(x = lifeExp,
                       y = gdpPercap,
                       colour = continent)) +
  geom_point() +
  geom_smooth() +
  scale_y_log10()

Adding size

Just like we mapped colour to a country’s continent, we can add size—the size of the points—to a variable (column) in our dataset. Let’s map size to population (pop).

  ggplot(data = gapminder,
         mapping = aes(x = lifeExp,
                       y = gdpPercap,
                       colour = continent,
                       size = pop)) +
  geom_point() +
  geom_smooth() +
  scale_y_log10()

Setting global rules

We have mapped variables to geoms. This means they will take a value (x, y, colour, etc) depending on their variable’s value (lifeExp, gdpPercap, continent, etc).

But what if we just wanted to set a rule? Say, what if we wanted the colour of all points to be blue? Or the transparency of all points to be 50% regardless of their country, lifeExp or gdpPercap?

Remeber that aes values are mapping values: they map a variable to a thing, and we keep those mappings in the aes() function.

If we want to set a rule outisde of an aesthetic, we do just that:

  ggplot(data = gapminder,
         mapping = aes(x = lifeExp,
                       y = gdpPercap,
                       colour = continent,
                       size = pop)) +
  geom_point(colour = "blue") + # colour = blue is outside of an aesthetic
  scale_y_log10() +
  labs(title = "Yay all of our points are blue!")

What would happen if we put colour = "blue" inside of aes? ggplot would do as it’s told: it would choose a colour for each value of the character string “blue”. As there is only one value, our chart looks like this:

  ggplot(data = gapminder,
         mapping = aes(x = lifeExp,
                       y = gdpPercap,
                       colour = continent,
                       size = pop)) +
  geom_point(aes(colour = "blue")) + # oh no, colour = blue is now INSIDE of aes().
  scale_y_log10() +
  labs(title = "Oh no! None of our points are blue")

But we got good information from colouring continents by colour, so let’s not throw that away. We can instead say “set the transparancy to 50%”. Transparancy in the ggplot-world is alpha. So we’ll set alpha = 0.5 to the geom_point geometry.

  ggplot(data = gapminder,
         mapping = aes(x = lifeExp,
                       y = gdpPercap,
                       size = pop,
                       colour = continent)) +
  geom_point(alpha = 0.5) + # added this to change transparancy, outside of aes().
  scale_y_log10() +
  labs(title = "That's some nice transparency")

Creating small multiples with facets

This is all getting a bit busy. It might be clearer to see each continent on its own separate chart. We can do this by adding a ‘facet’ to our plot: i.e. taking our chart, and plotting it ‘around’ ~ another variable.

# Define chart as an object
chart <- ggplot(data = gapminder,
                mapping = aes(x = lifeExp,
                              y = gdpPercap,
                              colour = continent,
                              size = pop,
                              label = country)) +
         geom_point(alpha = 0.5) +
         scale_y_log10() +
         facet_wrap(~continent) +
         labs(title = "GDP per capita and life expectancy, by continent",
              caption = "Source: Gapminder from https://gapminder.org")


# View the chart
chart

The code above says: take the chart we (now) know and love, and do that for each continent in our dataset showing each seperately.

A trendline might be nice, and we can do that by adding it to our chart object:

# Define chart as an object
chart_trend <- chart + 
               geom_smooth()

# View the chart
chart_trend

And, like we did above, we can make this plot interactive with ggplotly:

ggplotly(chart_trend)

1. Adding a new variable

First, we created a new column called gdp.

This uses the function mutate, which works like this:

# Add a new variable called 'newvar'
mutate(mydata, newvar = 10)

This means: take mydata and add a column newvar, which is 10 for every observation. See how we use one equals sign = to define something. You can read this as: newvar IS 10. (We’ll look at what two equals signs == means in the next section).

Thinking about the gapminder dataset, we could say that we wanted—for some reason—to make everyone richer with a everyone_richer variable that was the current GDP per capita gdpPercap \(\times\) 1000:

# To the 'gapminder' dataset, add a new variable called `everyone_richer'
mutate(gapminder, everyone_richer = gdpPercap * 1000)

## # A tibble: 1,704 x 7
##    country     continent  year lifeExp      pop gdpPercap everyone_richer
##    <chr>       <chr>     <dbl>   <dbl>    <dbl>     <dbl>           <dbl>
##  1 Afghanistan Asia       1952    28.8  8425333      779.         779445.
##  2 Afghanistan Asia       1957    30.3  9240934      821.         820853.
##  3 Afghanistan Asia       1962    32.0 10267083      853.         853101.
##  4 Afghanistan Asia       1967    34.0 11537966      836.         836197.
##  5 Afghanistan Asia       1972    36.1 13079460      740.         739981.
##  6 Afghanistan Asia       1977    38.4 14880372      786.         786113.
##  7 Afghanistan Asia       1982    39.9 12881816      978.         978011.
##  8 Afghanistan Asia       1987    40.8 13867957      852.         852396.
##  9 Afghanistan Asia       1992    41.7 16317921      649.         649341.
## 10 Afghanistan Asia       1997    41.8 22227415      635.         635341.
## # … with 1,694 more rows

Great—everone is richer! But note that this is not stored anywhere, the dataset with the everyone_richer variable was just printed on your screen. But we saw it do an important thing: for each observation, it took whatever the value of gdpPercap was and multiplied that value by 1000.

To get to the thing we were trying to do, add a gdp variable, we can use the mutate function and make sure we define it as an object:

# To the 'gapminder' dataset, add a new variable called `pop'
# Define this as 'gap_gdp'
gap_gdp <- mutate(gapminder, gdp = gdpPercap * pop)

As above, this will take the gapminder dataset and add a new variable gdp which is equal to each observations GDP per capita multiplied by its population.

To make sure this as all worked, we can print the head of our dataset:

# Look at the first few rows of `gap_gdp'
head(gap_gdp)

## # A tibble: 6 x 7
##   country     continent  year lifeExp      pop gdpPercap          gdp
##   <chr>       <chr>     <dbl>   <dbl>    <dbl>     <dbl>        <dbl>
## 1 Afghanistan Asia       1952    28.8  8425333      779.  6567086330.
## 2 Afghanistan Asia       1957    30.3  9240934      821.  7585448670.
## 3 Afghanistan Asia       1962    32.0 10267083      853.  8758855797.
## 4 Afghanistan Asia       1967    34.0 11537966      836.  9648014150.
## 5 Afghanistan Asia       1972    36.1 13079460      740.  9678553274.
## 6 Afghanistan Asia       1977    38.4 14880372      786. 11697659231.

And, like we have done many times before, we can visualise it:

# Plot 'gap_gdp' and define as 'this_plot'
this_plot <- ggplot(gap_gdp,
                    aes(x = lifeExp,
                        y = gdp,
                        colour = continent,
                        label = country)) +
            geom_point() +
            scale_y_log10()

# Look at 'this_plot'
this_plot

Note that here we have defined an object called this_plot with our plot, and then called this_plot by writing it.

We could make it easier to explore interactively by again using ggplotly:

# Use ggplotly to look at 'this_plot'
ggplotly(this_plot) 

2. Filtering to keep only observations from 2007

Next, we want to filter our dataset to only keep observations from 2007. i.e. we want to know what the state of the world was before the global financial crisis. To do this, we use the (suprise!) filter function. Our first argument is the dataset we want to do something to, and we follow that by a condition:

filter(original_data, [CONDITION])

10 == 20

## [1] FALSE

10 == 5 * 2

## [1] TRUE

x = 10
x == 2

## [1] FALSE

10 != 5 * 2

## [1] FALSE

gap_gdp07 <- filter(gap_gdp, year == 2007)

gap_gdp07$year

##   [1] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007
##  [15] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007
##  [29] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007
##  [43] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007
##  [57] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007
##  [71] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007
##  [85] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007
##  [99] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007
## [113] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007
## [127] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007
## [141] 2007 2007

unique(gap_gdp07$year)

## [1] 2007

3. Removing the year variable

We can choose and drop variables in our dataset using the select function. This comes in handy when you’re working with large datasets and your poor computer only has so much memory. Since we have already filtered our dataset to only include observations from 2007, we can drop the year variable.

gap_gdp07_noYear <- select(gap_gdp07, -year)

This reads: define gap_gdp07_noYear as the gap_gdp07 dataset and negatively select (drop) the year variable.

Wonderful! We have done the three things we wanted: we have one dataset that adds the gdp variable; the next that only keeps observations from 2007; and a final one that removes the year variable.

But, creating all these obscure datasets is odd. There is a better way.